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ABSTRACT 

The Education Data Improvement Project of the Council 
of Chief State School Officers (CCSSO) is described. The conceptual 
framework that guided its implementation is presented. The Education 
Data Improvement Project was designed to promote and facilitate the 
reform and refinement of the Center for Education Statistics (CES) 
national elementary and secondary public education database. As 
currently implemented, the common core of data is not adequate to 
accomplish its purposes in providing information about schools. The 
Project aims to improve the comprehensiveness of the national 
database and to improve the timeliness of reporting by state 
education agencies and the CLo. The guiding philosophy is to expand 
the existing Federal data systems and the data systems of the states 
to create a state-of-the-art national database. To do so, it is 
necessary to: (1) define existing systems; (2) identify changes to 
improve the systems; (3) identify basic and necessary statistics; and 
(4) establish content and providers for a national system that 
incorporates the best of existing systems. Four figures illustrate 
the Project goals and framework. (SLD) 
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PREFACE 



This document is one o£ a series of reports resulting from the Council 
of Chief State School Officers' Education Data Improvement "^uject. The 
project, funded by the u.S. Department of Education's Center for Education 
Statistics, is a joint effort of the states and the federal government to 
iinprove the quality and timeliness of data collected, analyzed, and 
reported by the Center. The project, initiated by the Council as the 
first effort of its State Education Assessment Center, coincided with the 
Department of Education's extensive redesign of the national 
elementary/secondary education statistical data system. Improvement of 
the Center's common core of data, collected annually from state education 
agencies, is the project's primary goal. 

In November, 1984» the Couw:!! of Chief State School Officers voted to 
"work actively with the National Center for Education Statistics 
(currently the Center for Education Statistics) to ensure that reporting 
of data from all sources is accurate and timely." This vote committed the 
Council to improving the comprehensiveness, comparability, and timeliness 
of data reported to the Center for Education Statistics by the state 
education agencies. 

In se/eral recent speeches and interviews, Chester E. Finn, Jr., the 
Assistant Secretary for the Office of Educational Research and Improvement 
COERI), listed four goals for strengthening the nation's ability to 
achieve educational excellence. The Department of Education's primary 
goal— to significantly improve the nation's educational statistical 
information base— both in the amount of data and its quality— suggests 
substantial interest in the work and goals of the Education Data 
Improvement Project. 

The Center for Education Statistics and the states share 
responsibility for a statistical system in education that is inadequate 
for today's needs. This project is one effort wherein they are working 
togethex to make the basic system efficient and effective. 

The goals of the project are to describe state collection of data 
elements currently contained in the common core of data, to describe those 
elements that might be added to make the common core of data adequate and 
appropriate for reporting on the condition of the nation's schools, and to 
make recommendations to states and the Center for Education Statistics for 
making the common core of data more comprehensive, conqjarable, and 
timely. During this first year of the project the focus has been on the 
school and school district universe files. 

The project is examining the universe files to identify all states 
collecting specific data elements, to specify in detail the definitions 
and specifications used by each of the states for each data element, and 
to isolate discrepancies in ways different states define and measure those 
various elements. This current report presents a description of the 
Education Data Improvement Project and the conceptual framework that 
guided its ioqplementations. 
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iNraoDucrioN 



In November, 1984, in a dramatic break with its own historical 
position, the Council o£ Chief State School Officers voted to . . 
provide leadership in the develoixnent , use, and interpretation of 
assessment and evaluation procedures and results.'' This vote ':ommitted 
state education agencies and the Council of Chief State School Officers to 
state-by- state comparisons of eaacational progress, a commitment which 
carries an enormous responsil i li ty for adequate and appropriate 
collection, analysis, and reporting of information. 

The decision taken by the Council resulted in a special unit, the 
State Education Assessment Center, to implement this and related policies 
of the ciiief state school officers. The Center was put into operation in 
September, 1985, and has been carrying out intensive and comprehensive 
conceptualization and planning for this major new collective enterprise of 
the state education systems. 

The Assessment Center has responsibility for providing national 
leadership on educational assessment and evaluation. The Center 
represents the states' elementary and secondary education leaders in 
developing, interpreting, and using assessment and evaluation procedures 
and results for reporting on the status of schools and schooling. The 



Center involves every state in the planning and procedures o£ its work so 
that all perspectives are represented. Technical and user ad- isory 
rietworks ensure the end results are accurate and responsive to state 
needs. 

The Assessment Center's resonsibilities include leadership in 
collecting and reporting comprehensive, timely evaluation and assessment 
information so state-by-state comparisons can be made. For valid and 
reliable information, the Assessment Center must seek education statistics 
and indicators that are comprehensive, standardized across all states, and 
reported in a timely way. 

The State Education Assessment Center is charged with implementing the 
chief state school officers' "Position Paper and Recommendations for 
Action," adopted at their 1984 Annual Meeting. The "Position Paper" 
directs the Assessment Center," to work actively with the National Center 
for Education Statistics [currently called the Center for Education 
Statistics] to ensure that reporting of data from all sources is accurate 
and timely." In response, the Council, under contract with the Center for 
Education Statistics, undertook a comprehensive, three-year effort to 
improve, expand, and standardize the Center for Education Statistics' 
elementary/secondary public education database. The Education Data 
Improvement Project is the central, or cornerstone, effort by the State 
Education Assessment Center to upgrade and standardize basic statistics in 
education at the state and national levels. 

This report provides an introduction to, and an overview of, the 
Education Data Inq^rovement Project. The report is divided into three 
major sections: a project overview and organizational structure, a 
discussion of the conceptual framework that guides both the design and the 
operation of the project, and the a description of the technical approach 
of the project. The report concludes with a discussion of uses and 
implementation of the project's outcomes and recommendations. 

This report, 'Technical Report: Conceptual Framework," is the first 
in a series of documents that describe the Education Data Improvement 
Project, its findings, and recommendations. The "Technical Report" is 
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acconpanied by two additional descriptive reports: the "Development of a 
Shuttle for Verifying Data Elements Collected by State Dep*>rtraents of 
Education and Reported to the U. S. Department of Education's Center for 
Statistics," and "A Compendium: State Profiles of School and School 
District Universe Files." 

The findings and recommendations from the first year of the project 
are presented in a series of white papers. Tiie white papers are as 
follows: 

(1) Summary: State Collection Practices on Universe Data Elements, 

(2) School and Student Classifications for Universe Data Files, 

(3) Variations in Definitions and Procedures for Student Counts: 
Enrollment, Fall Enrollment, Membership, and Average Daily 
Membership, 

(4) Variations in Definitions, Counts, and Reporting of 
Prekindergarten, Ungraded, Homebound, Educated-at-Home, and 
Special Education Students, 

(5) Collecting National Statistics on Dropouts, 

(6) Federal Programs Information on School and School District 
Universe Files, and 

(7) Summary: Recommendations for Improving the National Education 
Statistical Database. 

The final white paper, "Recommendations," summarizes the major 
recommendations of the other white papers. It contains recommendations 
for the overall national statistical system, including specific 
recommendations and considerations for the Center for Education Statistics 
and the Center's Common Core of Data and specific recommendations for 
consideration by states. The "Recommendations" paper identifies critical 
terms that must be standardized if the national system is to be comparable 
across states and specifies new definitions for those terms. Finally, 
this white paper contains a list of the project's recommended data 
elements for the universe files of the national elv^mentary and secondary 
public education statistical database. 

In succeeding years of the project, reports and recommendations will 
be produced on core fiscal data and on non-fiscal, non-universe data 
reported on schools. 




PROJECT OVERVIEW AND ORGANIZATOKAL STRUCIURE 



The Education Data Improvement Project, a joint effort of the Council 
of Chief State School Officers and the U. S. Department of Education's 
Center for Education Statistics is the first effort of the State 
Education Assessment Center. It is designed to promote and facilitate the 
reform and refinement of the Center for Education Statistics' national 
elementary and secondary public education database. The project's primary 
focus and the basis for final recommendations is education statistical 
data from state education agencies collected annually through a set of 
instruments currently called the conmon core of data. The objectives of 
the Common Core of Data drive the project's work. 

There are four primary objectives for the core natioiial education 
statistical database. They are (1) to provide the official listing of all 
schools (approximately 87,000) awd sdiool districts (approximately 16,000) 
in the country's public education system, (2) to provide basic descriptive 
information on the nation's schools and schooling, (3) to provide 
information on the financing and costs of schools and schooling, and (4) 
to provide a sampling frame for major national studies on education. 

As currently implemented by the Center for Education Statistics and 
states, the common core of data is inadequate to accomplish its stated 
purposes. Traditionally available data have been reported three or four 
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years after the date of collection and, as a consequence, have not been 
tiaely for nost federal and state decisionmaking cycles. The current 
daubase does not contain enough information for selecting saiiq;)les for ths 
Center's om national survc/s and studies of schools and schooling. The 
school universe file, for example, contains the following information: 

e school name, address, and telephone number, 

• school type (regular, special education, vocational, other), 

e grade span, 

e nuiber of teachers (total), and 
e number of students (total). 

There is no information about the types of stut'snts served (e.g., 
racial/ethnic grouping, sex, number of limited English proficient 
students) or the type of locale of the school (e.g., urban, rural). The 
available data, while' u>scriptiv3, do not provide minimum information that 
should be known about the nation's schools. 

During the first project year, the Education Data Improvsment Project 
addresses the i?^inement and enhancement of the school *.id school district 
universe files to meet the objectives of providing a listing of schools 
and districts, providing basic education statistics, and serving as a 
national sampling frame. The purpose of the first project year is to 
improve the school and school district universe files. The second year of 
the project focuses on fiscal data. The third project year continues the 
work of the first two years to integrate the universe files and general 
descriptive statistics with the education finance data into a 
cooq>rehensive national statistical data system. 

The goals of the project are to improve the comprehensiveness of the 
national database to more adequately meet its objectives, to increase the 
comparability of data reported by the 50 states, 6 extra- jurisdictional 
territories, and the District of Columbia, and to improve the timeliness 
of reporting by boili state education agencies and the Center for Education 
Statistics. 

The guiding philosophy for the project is to build onto the extensive, 
existing data systems of the Center for Education Statistics and the state 
education agencies and to use the strongest elements within those systems 
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to cret.te a state-of-the-art national statistical database. To implement 
this philosophy, four major activities are necessary: (1) to define, 
operationaily, the existing systems and identify elements that are similar 
across states and the Center, (2) to identify changes in the system that 
will iafTove its effectiveness and efficiency, (3) to identify the basic 
statistics necessary for a revised, comprehensive national statistical 
system, and, (4) based on knowledge of the current systems, establish 
content and procedures for a national system that incorporates the best of 
the individual systems and establishes the "best fit," lAere possible, 
across all of the systems. 

Organizational Framework and Relationships 

This philosophy of redesigning, refining, and standardizing the 
existing statistical system and the organizational structure within which 
the project operates determines, in great part, xJie project's procedures 
and processes. The organizational structure places the project within a 
network of related agencies that share responsibility tor a comprehensive, 
comparable and timely national statistical system. These agencies include 
states, the Center for Statistics, the Council of Chief State School 
Officers. Figure 1 graphically illustrates the organizational 
interrelationships and the project's position within them. The project 
coordinates the various organizations to implement the project processes 
and to iaq>lement the activities described above. 

Structure 

The project is funded by an U. S. Department of Education's Center for 
Education Statistics award to the Council of Chief State School Officers 
and is administered by the State Education Assessment Center within the 
Council. 

Center for Education Statistics . The federal mandate for the Center 
for Education Statistics, in the U. S. Department of Education's Office of 
Educational Research and Iii5)rovement, is to report periodically on the 
condition of education in this country. The Center implements its charge 
through two general processes: by maintaining a national statistical 
database on schools and schooling, and by conducting periodic assessments 
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Figure 1. Organixational Framework for the Education Data Improvement Project 
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and surveys of education progress. For the past decade, the statistical 
database has consisted of the common core of data reported by the 57 state 
education agencies for all of the country's schools and school districts. 
The National Assessment of Educational Progress (NAEP), conducted every 
two years in reading, mathematics, science and social studies, evaluates 
and reports on student progress in these primary subject ares^. Periodic 
studies on specific topics are conducted. Some recent examples are the 
High School and Beyond study, the National Educational Longitudinal Study 
of the Class of 1988 (NELS 88), and the School and Staffing Survey. 

There are several components of the national statistical database, 
such as elementary and secondary public, noi^blic schools, and post 
secondary education. The target of the Education Data Improvement Project 
is the national education statistical database on elamentary and secondary 
public schools. 

Couixil of CMef State School Officers . The Council of Chief State 
School Officers is a membership organization representing the leadership 
of the nation's 57 state education agencies. The members of the Council 
represent the 87,000 schools and 16,000 school districts making up the 
national public education statistical database. The "Position Paper" 
adopted by the Council in 1984 provides a public statement of its position 
on national statistics on education. The "Position Paper" states: 

... The publication of data brings education to the attention of the 
public and makes it an issue of enduring concern. By comparing the 
performance of a school, district, or state with itself over time, or 
with other schools, districts, or states, data serve to exhort, 
motivate, or reward. 

A second use [of data] is to aid local, state and national 
policymakers in understanding the consequences of changes in policy 
and to aid them in iii5)lementing policies once they are adopted. . . . 

Finally, and coming full circle, data serve to make education 
accountable to the public. . . . 

The major challenge for the Council and its membership is to provide 
leadership in designing a statistical data system— an education indicators 
framework and a student assessment program- -lAich allow for efficient use 
of data for decisionmaking at school, district, state, and national 
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levels. The Education Data Improvement Project focuses on the education 
statistics component of this challenge. Improved national statistics will 
enhance the Council's efforts for better education indicators. 

States . States share responsibility for the content and quality of 
the national statistical database. They provide the data and are usually 
the unit of analysis reported by the Cer.tsr for Education Statistics. The 
Center for Education Statistics reportji are used, in turn, to make 
judgements about the relative efficiency and effectiveness of individual 
state education systems. States have vested interests in tlie accuracy and 
cooparability of the data in the national statistical system. 

Interrelationships 

The Center for Education Statistics is a federal agency with no 
program authority or responsibility for education. It is charged with 
reporting on the condition of education in the country. State education 
agencies are charged with responsibility for the education oi children 
within their jurisdictions. They have varying amounts of authority over 
the local school districts making up their systems. They collect 
statistical data from the local school districts to account for 
educational processes and progress within their own borders. They report 
those statistics, as requested, to the Center for Education Statistics. 

The Council of Chief State School Officers serves as an intermediary 
between the states and the Center for Education Statistics, having 
affirmed its responsibilty to provide leadership for improving the 
national education statistics. The Council, as part of its charge, seeks 
to improve the Center for Education Statistics database. The State 
Education Assessment Center is the unit in the Council administratively 
responsible for improving education data. The Assessment Center, through 
the Education Data Improvement Project, seeks to influence the quantity, 
quality, and timeliness of basic data on public elementary and secondary 
schools through the project. 

The Education Data Improvement Project coordinates all of its 
activities with state education agencies. Project recommendations are 
conveyed simultaneously to states and to the Center for Education 
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Statistics for their consideration. The Center for Education Statistics 
may choose to incorporate the recommendations into its definitions and 
standards. States, working cooperatively with the Center for Education 
Statistics, will devise strategies for responding to changes on an 
individual basis. 

Summary 

The process of improving the national education statistical system 
involves independent entities participating in an enterprise with no 
foimal commitments for those individual entities to change. Two factors 
hold the entire process together and provide the basis for an improved 
system and better national data on public schools and schooling: all 
parties are committed, even if not formally, to comprehensive, compi^rable, 
and timely education data, and all parties believe that change in the 
system(s) must be accomplished to achieve their c(»omitments to improved 
data. 
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GONCEPTUAL FRAMEWORK 



The Education Data Improvenient Project's final products are 
recoDmendations to the Center for Education Statistics and to states for 
improving the conq)rehensiveness, comparability, and timeliness of data in 
the national statistical database. The reconmendations are intended to 
make the database both adequate and appropriate for achieving its four 
purposes of providing (1) the official listing of public elementary and 
secontiary schools and school districts, (2) general descriptive statistics 
on schools and schooling, (3) general data on the financing of pubic 
education, and (4) a sampling frame for najor national studies. 

To achieve the desired outcomes efficiently and effectively requires 
formulation of a conceptual framework to guide the design and 
implementation of project processes. A conceptual framework specifies 
purposes of the study, tasks to be accomplished, meiiiodology of the study, 
data sources, and desired outcomes. A project's technical approach should 
derive from a sound conceptual framework. Figure 2 is a graphic 
presentation of the conceptual framework for the Education Data 
lBq)rovement Project. 
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Figure 2. Conceptual Frameworic for the Education Data Improvement Project 
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Purposes of the Study 

The purposes of the study are: 

1. to conceive and describe an ideal system for basic statistical data 
collection and reporting tor elementar)' and secondary public 
education; 

2. to describe existing state and national education statistical data 
systems; and 

3. to reconcile the existing systems with the ideal to create a new 
system that is technically accurate and feasible for the Center for 
Education Statistics and state education agencies. 

Tasks of the Study 

To fulfill these purposes, the study is organized around six major 
tasks : 

1. Identify components (i.e., structures, processes, and data 
elements) necessary for a comprehensive national data system that 
will provide (a) a complete listing of all public schools in the 
country, (b) basic statistical data on schools and schooling, (c) 
basic data on the financing and costs of education, and (d) a 
national sampling frame. 

This task runs parallel to, and concurrent with, all other tasks in 
the Project. It both informs and is informed by other tasks. 

2. Describe the current statistical data systems of the Center for 
Education Statistics and states, including specific data elements 
that are collected and reported by each. 

3. Identify terms and data elements in both the ideal system and the 
current systems that oust be coiq)atable for this system to be 
sound. 

4. Coaq)are state and Center definitions and specifications to 
determine areas of non-comparability. 

5. Reconcile differences between the existing systems and the proposed 
ideal system. 



These tasks are not necessarily secpjential. The study is 
"self -informing," with the responses to one task or set of tasks requiring 
revisitation of previous ones. Identification of specific data elements 
and states' current uses of them, for example, require that previously 
identified data elements be reexamined to determine if they continue to be 
necessary. These tasks form a complex of decision points, each informing 
the others until a "best fit" solution is identified. 
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Methodology 

As with the research questions, the methodologies employed in the 
Project are interrelated and self*in£orming. Three major types of 
strategies are used: explication and description, inquiry, and 
reconciliation. In each case, both qualitative and quantitative methods 
are employed: quantitative methods yAnen it is possible to obtain numbers 
of definitions or uses of a term; qualitative methods to generate ideas 
and explain findings. 

Qualitative Methods . Qualitative methods are generally employed at 
the beginning of a strategy and at the end. At the beginning, qualitative 
methods are used to search available irJormation to create pools of 
possibilities that are then catalogued for systeicatic analysis. In many 
cases, qualitative analyses are used to examine current statistical 
systems in other agencies, such as the Office of Special Education and 
Rehabilitation Services in the U.S. Department of Education and the Bureau 
of the Census, to determine their uses of collection and reporting 
procedures or their definitions of terms or specifications for individual 
data elements. Knowledge of other systems assists in making choices 
within the national education statistical database to select most likely 
choices for further study. Such prior knowledge is critical to efficient 
operation of project processes. It permits the project to narrow the 
options to be examined in greater detail or to extend the analysis. 

At the end of a project strategy, when quantitative information is 
available for making decisions, qualitative methods are again employed to 
help explain the findings. Quantitative tabulations of state responses to 
a data element or term often obscure the profundity or pervasiveness of a 
given problem or impact that an individual state variation will have on 
measurement across all states. For example, a state may define a dropout 
to be a student above compulsory attendance age only, or below compulsory 
attendance age only. While only two states use coapulsory attendance as a 
factor in counting students }Aio drop out of school, the problem must be 
identified and resolved if data are to be comparable across states. 
Qualitative analysis helps to identify and explain such issues. Without 
using various qualitative techniques many subtleties imbedded in 
quantitative data will be missed. 
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(Xantitative Methods . (^titative procedures are employed 
throughout, whenever possible. When qualitative methods identify a pool 
of options, the options are coded on a semantic tree network so the 
individual definitions and variations can be compared. Figure 3 presents 
the coiiq>onents of the semantic tree network for a specific data element 
"public school student." For example, individual states have regulations 
or guidelines that govern their uses of a term or data element, such as 
enrollment counts of public school students. Some states include in their 
enrollment counts all students who withdraw; while other states exclude 
withdrawal from their enrollment counts. The latter enrollment count is 
the same as the membership counts in other states. The codification of 
state responses according to a standard, semantic framework allows such 
coa|;arisons to be made and quantified. Quantification expendites 
manipulation and interpretation. 

State responses are quantified to ensure that judgements are made 
based on the current status of data collection and reporting in the 
states. Patterns in quantifiable data are easier to detect and explain. 

Explication and Description . For the purposes of this study, 
explication is defined as careful and systematic study of facts, 
revelation of existing and past conditions, and making those conditions 
explicit. The project's purpose of describing existing state and national 
data systems requires review of existing documents from the Center tor 
Education Statistics and states, including data collection forms, data 
element dictionaries, guidelines and regulations. As an economy measure 
for saving time and project and state staff resources, the project's 
explication processes eliminate the need to examine all possible 
components in existing state systems. For example, the conmon core of 
data does not request enrollment by grade at the school level; however, 
through an early exploratory telephone interview with states, it was 
discovered that all states currently maintain that information in their 
systems. On the other hand, examination of information from the Center 
and from statef indicates that the current use of standard metropolitan 
statistical area (SMSA), designation by district, is neither accurate nor 
ade<9iate for its intended purposes. Consequently, this data element was 
not included in the project's inquiry stages. 
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Data Elenent 



a definite, variable, Measurable unit of schools, 
students and staff (identified by all capital lettcirs 
and bold type) 



Specification — detailed description of neaning and critical 

factors affecting aeasurenent of a data element as 
defined and agreed to by the Center for Statistics 

1. kernel — principal part of the specification; part 

on which everything else rests (identified 
by initial capital letter and bold type) 

2. base string — essential elewnt prescribing limits 

of a specification (identified by 
black bullets) 

factor identified by state(s) as operationally 
lacking agreement with a specification (separated 
from base strings by a line; identified by 
white bullets) 



Variant String 



DATA ELBIENT AND SPECIFICATIONS 

Data Element PUBLIC SCHOOL STUDBfT 

Kernel Individual enrolled in a school: 

Base String • operated by a local public school agency 

• operated bf a state education agency 

• operated b/ federal government 

Variant String o excludes nonresident students 
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Figure 3. Components of Semantic Tree Networic 
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The coanon core of data has been in existence for more than a decade 
and was preceded by Owwr national data systems; each state education 
agency has an extensive statistical data system. This project builds on 
ind refines Uiose existing systems and is not intended to create a new 
one. Thus, discovering the components that currently exists in the 
national and sUte systems and their subcomponents, their actual and 
perceived uses, and their interrelationships is a necessary first step. 
However, the explication uethodology is not just a preliminary phase of 
the study; it is necessary at each step to ensure that all of the facts 
are known to focus further study on the most pertinent aspects of the data 
systems. 

While the explication steps inform subsequent stages of the process, 
those subsequei;t stages reveal information that makes further explication 
necessary. This is a cyclical and continuing process throughout the 
project. 

The explication and description methodologies ara primarily 
qualitative in nature. Wherever pos lible the qualitative information is 
codified and cjuantified to assist in interpretation. 

Inquiry . The processes enqployed in the project are based on the 
tenets that the quality of information is significantly improved if it is 
based on aggregate judgements of a number of individuals. Miiltiple 
opportunities for an individual to correct and refine given responses, 
significantly iiiq>rove the quality of information. The methodology used 
for the inquiry phase of the Project is an "inverted Delphi" technique. 

The Delphi technique was inr^oduced in 1963 by the Rand Corporation as 
a process of multiple iteratio»:c with controlled feedback. It is a method 
"for the systematic solicitatiou and corroboration Oj.' judgements on a 
particular topic through a set of carefully designed sequential 
questionnaires interspersed with summarized information and feedback of 
opinions derived from earlier responses." The Delphi technique is 
gemrally used i.o establish consensus about a particular issue or set of 
issues. In the Education Data Improvement Project, the technique was used 
to refine, successively, information from the state education agencies and 
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to iq(>TOve the q^tlity of the information. Thus, it was not used to 
generate consensus but rather to differentiate state practices that are 
initially presuned to be similar. Thus the ''inverted Delphi", 

The inquiry phase of the project is structured by multiple iterations 
of data collection. The first iteration is a summary of each state's 
practice, based on an initial ex|lication derived from state forms and 
report. State data collection and reporting forms are reviewed by project 
staff 9 findings are preprinted onto instruments sent to state respondents 
who, in turn, respond with more accurate information. Individual state 
respondents are then provided opportunities to react by concurring with 
the finding, adding or subtracting information from the finding, or 
submitting new information altogether. 

Each successive iteration results in a wider range of responses and 
forms the basis for the r«cxt iteration(s). As each set of responses is 
received from states, individual item responses are codified, using a 
semantic tree network, and the codified responses are returned to the 
respondents. In subsequent iterations, individual respondents refine 
their initial responses, using the pool of knowledge from all 
respondents. In this manner, state information and manipulatable 
interpretations are refined and the overall pool becomes more complete. 
Figure 4 illustrates the coding of the term "public school," The bulleted 
word strings ar^ codified and manipulatable responses from all state 
education agencies, and the aggregate of the word strings form the 
project's database for the reconciliation phase that follows. 



TmH AND DEFINITION 



Term PUBLIC SCHOOL 

Kernel Elementary or secondary school: 

Base • operated by a state 

String e operated by a subdivision of a state 

e operated by a government agency within a state 
e operated by the federal government 



Variant o operated by an administrative unit subject to 

String state rules, regulations and minimum standards 

r excludes publicly siq;>ported private agency school 



ire 4. Example of Semantic Coding of "Public Schoof 
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Reconciliation . The project's database is structured around specific 
terms, used by the Center for Education Statistics and states, that 
influence the statistical and financial data reported by states to the 
Center. If the terns are defined differently by different states, then 
data across those states will not be comparable. For example, the term 
"school" is the basis for all student counts. If one state includes or 
excludes a class of schools that is assumed by the Center to be included, 
then that state's school and student counts will be either inflated or 
deflated. This not only provides an inaccurate and incomplete picture of 
the number and types of schools, it also affects all other calculations 
using those counts, such as costs per student and student-teacher ratios. 

The project's database contains, in addition to measurement ^related 
terms, data elements that are considered potential statistics for the 
national statistical data system, i.e., student counts, number of limited 
English proficient students, number of students by racial/ethnic grouping, 
among others. The individual state specifications identify variations in 
the collection and reporting of data that affect coiq>arability of 
statistics across sutes. For example, when a student count is taken 
influences the numbers. Enrollment, viewed as a cumulative count of all 
students registered for schooling, is a different population of students 
if it is taken in the fall as coiiq}ared to a count taken in the spring. 
The fall count includes a minimus of students vAo have transferred in or 
transferred out, while a spring count includes all of the students 
transferring in without adjusting for those students transferring out. A 
spring enrollment c unt reflects duplicate reporting of all transferred 
students. 

Reconciliation of information within the Project's database takes two 
foims: tabulation of different responses (i.e., word strings) by 
individual states and across states, and interpretation by "experts." The 
tabulations provide the basis for the interpretative processes. The 
interpretative process either reconciles the differences, based on 
available information or results in new data collection and tabulations, 
to provide more complete studies of the current state practices. The 
"experts," recognized by other authorities as knowing the field and 
generally recognized as knowing the particular facts and findings, are 
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convened in task forces and study groups to examine particular areas of 
c<H)cem in how data elements are defined. For exaiq>le, a task force of 
state and local education agency staff with experience in counting 
dropouts, federal agency and private enterprise researchers, with 
responsibilities for evaluating and reporting national dropout statistics, 
and university-based researchers examined current state practices and made 
recoonendations for developing a standard dropout statistic across all 
states. 

Data Sources 

All lm\>e methodologies— explication, inquiry, and reconciliation- 
employ the use of those persons closest to, and responsible for, the 
nature an*, quality of state and national data. Those persons serve as 
respondents for the project. At the national level. Center for Education 
Statistics' administrators :^id staff provide data for project use. At the 
state level, the primary respondents are persons identified on the state's 
contract with the Center as the common core of data coordinator. Also at 
the state level, the chief state school officer's designee to the 
Council's CooDittee on Evaluation and Information Systons (CEIS) provides 
a review of all information reported for that state. All final 
information is reviewed by the chief state school officer. 

The data sources for the Project are documents used by the Center and 
individual states for collecting and reporting data to the Center for 
Education Statistics. Documents include the Center's handbook series on 
the coomon core of data, their instruments for collecting data, and 
guidelines and regulations pertaining to the data collection process. For 
states, the documents generally include state legislation, handbooks, 
guidelines, and regulations for state and school district reporting, data 
element dictionaries, and data collection and reporting forms. The 
coiiq>osition of documents varies rrom state to state. The Project uses 
whatever is available from individual states. 
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Technical Approach 



Following is a description o£ nine major strategies used by the 
project to accooplish its tasks and to arrive at final recommendations for 
the Center for Education Statistics and states. While the format suggests 
that the strategies are sec^ential , they are not. The strategies are 
interrelated and overlap one another forming a complex of technicpies used 
by the project. Two companion reports, "Development of a Shuttle for 
Verifying Data Elements Collected by State Departments of Education and 
Reported to the U. S. Department of Education's Center for Statistics" and 
"A Compendium: State Profiles of School and School District Universe 
Files" describe the project's use of the strategies. 

• Review Center for Education Statistics documents , including all 
parts of tlie Common Core of Data instrument used during the 1985-86 
school year. There are six parts (i.e., school universe, school 
district universe, local education agency fiscal and non-fiscal, and 
state fiscal and non^-fiscal). The project is designed to address 
the two universe parts during the first project year with non-fiscal 
elements added as necessary to provide basic descriptive statistics 
on schools and school districts. The second project year focuses on 
fiscal data with additional non-fiscal data as needed to provide 
coiq)rehensive financial information. The third year of the project 
integrates findings of the universe files, non-fiscal and fiscal 
components into an integrated, coB^rehensive statistical data 
system. 

Other documents used include recent guidelines for completing the 
COTBron core of data instruments, appropriate Handbooks in the State 
Education Records and Reports Series, invited papers for the 
Elementary and Secondary Statistics Redesign Project and other 
Center reports, such as the "Condition of Education," and "Digest of 
Education Statistics." An initial review yields an extensive list 
of potential universe data elements, either currently collected by 
the Center or viewed as necessary by users of the national 
statistical data system. (Potential users are units within the 
Center itself for selecting samples for the National Assessment of 
Educational Progress or NELS 88.) The initial listing of data 
elements includes all elements mentioned in the literature, with no 
judgements regarding their appropriateness or adequacy for the 
universe files. 

Reviews of Center documents provide accepted definitions of terms 
and specifications for selected data elements. These definitions 
and specifications become initial standards by which state responses 
are measured. Variations from the standards form the basis for 
reconciliation stages of the Project. 
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Interview individuals on the Center for Statistics* staff and on 
state educat ion agency staff who are familiar with the Center and 
the GooBon Core of Data instruments and data collection procedures. 
The interviews provide evaluations of each element on the list of 
potential data elements and specific recommendations for including 
then, or not. In many instances the interviews suggest additional 
or related terms to be standardized, if data elements are to be 
comparable across states (e.g., school, public school). 

The interviews provide initial and continuing reviews and 
refinements of the standard definitions. Interviews occur at the 
end of the inquiry stages and within the reconciliation stages to 
interpret findings and individual state variations. 

Evaluate state education agCTcy instruments used for collecting data 
frr school districts. The evaluations identify data elements 
currently and generally collected by states. These additional data 
elements and terms are evaluated for possible inclusion on the 
master list. Also, the evaluations identify data elements on the 
original list that are not generally included by states. In such 
instances, the data elements are re-evaluated for determining their 
continued inclusion. The list of data elements is further refined 
using both interview information and evaluations of guidelines for 
completing the instrument and other related state- specif ic 
documentation. Related terms are identifed and added to the list 
in a similar manner. 

Develop an instrument for verifying data elements collected by state 
departments of education ah3 reported to the Center for Statistics. 
The instrument, or Shuttle, is coiq)leted by Project staff using 
state data collection forms, sent to the states, and verified or 
corrected by Common Core of Data Coordinators in each state agency. 
The Shuttle process is described in a \Aiite paper, *n)evelopment of a 
Shuttle for Verigying Data Elements Collected by State Departments 
of Education and Reported to the U. S. Department of Education's 
Center for Statistics." 

Develop a taxonomy for coding state responses on the Shuttle . The 
taxonomy is a semantic tree network that standardizes language 
across all states and provides a format that permits comparisons of 
individual state definitions and procedures. The newly-worded and 
formatted responses are verified by states to assure the intent of 
their original , open-*ended responses is maintained. The state 
respondents are provided with a cumulative listing, or catalogue, of 
all state responses, to compare information across states. The 
catalog assures that factors omitted by one state from the responses 
to the Shuttle but included by another are considerr^d by both states 
to ensure the continued accuracy and coiLpleteK'^ss of initial 
responses. 

Develop individual state profiles of selected data elements and 
relatea terms from the responses on the Shuttle, using the taxonomy 
described above. The Profiles display contextually, the terms and 
data elements presented on the Shuttle, incorporate changes 
identified through the Shuttle process, and permit another 
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opportunity for state respondents to review and refine information. 
The Profiles are corrected, based on the first review, and are 
submitted for verification at least one more time. After the state 
respondent is satisfied that information is complete and accurately 
represented, the Profiles are submitted to chief state school 
officers for final verification. After the chief state school 
officers review and refine the Profiles, information on the 
individual state Profiles becomes the database for the 
reconciliation stage of the Project. 

The Profile process and a complete set of Individual State Profiles 
is presented in "A Conq^endium: State Profiles of School and School 
District Universe Files." 

• Conduct telephone surveys of Common Core of Data Coordinators to 
obtain botn clarifications of state information on the Profiles and 
additional data not received through the Shuttle or Profile 
processes. 

• Convene task forces and study groups to provide in-depth analyses of 
data elements or sets of el^nts, to reconcile differences observed 
across states on terms and data element specifications, and to 
suggest recommendations for iDq)roving definition and collection on 
the data element. 

• Prepare preliminary analyses of state-by-state data for verification 
of results by state respondents and chief state school officers. 
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REC(»flENDATIONS: PRESEWTATION AND IMPLBiENTATION 



The Education Data Improvement Project conveys its findings and 
recommendations conjointly to the Center for Education Statistics and the 
Council of Chief State School Officer's Committee on Coordinating 
Education Information and Research (CEIR) for action. CEIR receives 
project recoranendations and may choose to take action based on the 
Council's policies on evaluation and assessment. CEIR may approve the 
recommendations, modify them, or disapprove them. CEIR reports its 
recommendations to the Council of Chief State School Officers. Council 
actions and positions are forwarded to the Center for Education Statistics 
and to states. StaCes take the Council's recommendations under 
consideration for possible implementation with the Center for Education 
Statistics. 

The Center for Education Statistics may choose to incorporate the 
recommendations into its elementary and secondary education statistical 
system. If the Center chooses to make recommended changes, it will 
coordinate those changes with individual state education agencies. 
Participation by states is, of course, voluntary. 
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